An Iterative Data Collection Approach for Multimodal Dialogue Systems
نویسندگان
چکیده
This paper deals with the way in which data for multimodal dialogue systems are collected. We argue that for multimodal data, an iterative data collection strategy should be followed. Instead of a single major data collection effort using a “Wizard of OZ” (WOZ) or “prompting” experimental setup, several smaller data collections should accompany the system development. We also describe the “script” experimental setup we developed. It is in between the WOZ and prompting setup, and can be used as a cost effective design for the first data collection within the iterative data collection strategy.
منابع مشابه
A Corpus Collection and Annotation Framework for Learning Multimodal Clarification Strategies
Current dialogue systems are fairly poor in generating the wide range of clarification strategies as found in human-human dialogue. The overall aim of this work is to learn when and how to best employ different types of clarification strategies in multimodal dialogue systems. This paper describes a framework for learning multimodal clarification strategies for an in-car MP3 music player dialogu...
متن کاملEvaluating Dialogue Strategies in Multimodal Dialogue Systems
Previous research suggests that multimodal dialogue systems providing both speech and pen input, and outputting a combination of spoken language and graphics, are more robust than unimodal systems based on speech or graphics alone (Andr ́e, 2002; Oviatt, 1999). Such systems are complex to build and signifi cant research and evaluation effort must typically be expended to generate well-tuned modu...
متن کاملAn Adaptive Approach to Collecting Multimodal Input
Multimodal dialogue systems allow users to input information in multiple modalities. These systems can handle simultaneous or sequential composite multimodal input. Different coordination schemes require such systems to capture, collect and integrate user input in different modalities, and then respond to a joint interpretation. We performed a study to understand the variability of input in mul...
متن کاملHow was your day? An architecture for multimodal ECA systems
Multimodal conversational dialogue systems consisting of numerous software components create challenges for the underlying software architecture and development practices. Typically, such systems are built on separate, often preexisting components developed by different organizations and integrated in a highly iterative way. The traditional dialogue system pipeline is not flexible enough to add...
متن کاملCollecting mobile multimodal data for match
Next-generation multimodal systems designed for use in mobile environments present challenges to the task of data collection that are not faced by speech-based systems. We discuss some established data collection and evaluation methods and their limitations in the context of a mobile multimodal system. These limitations are addressed by the “on-board” multimodal data collection method developed...
متن کامل